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Abstract — Image re-ranking, as an effective way to improve 
the results of web based image search, has been adopted by 
current commercial search engines. Given a query keyword, a 
pool of images are first retrieved by the search engine based on 
textual information. By asking the user to select a query image 
from the pool, the remaining images are re-ranked based on 
their visual similarities with the query image. A major challenge 
is that the similarities of visual features do not well correlate 
with images’ semantic meanings which interpret users’ search 
intention. Experimental results show that 20% to 35% relative 
improvement has been achieved on re-ranking precisions 
compared with the state of-the-art methods. 


Index Terms — Re-ranking, Query semantic space, Text 
based search , Ranking protocol. 

I. INTRODUCTION 

Web-scale image search engines mostly use keywords as 
queries and rely on surrounding text to search images. It is 
well known that they suffer from the ambiguity of query 
keywords. For example, using “apple” as query, the retrieved 
images belong to different categories, such as “red apple”, 
“apple logo”, and “apple laptop”. Online image reranking has 
been shown to be an effective way to improve the image 
search results. Major internet image based image re -ranking. 
According to our empirical study, images retrieved by 120 
query keywords alone include more than 1500 concepts. 
Therefore, it is difficult and inefficient to design a huge 
concept dictionary to characterize highly diverse web images. 

II. RELATED TECHNOLOGY PRINCIPLE 

A. Existing system 

This is the most common form of text search on the 
Web. Most search engines do their text query and retrieval 
using keywords. The keywords based searches they usually 
provide results from blogs or other discussion boards. The 
user cannot have a satisfaction with these results due to lack of 
trusts on blogs etc. low precision and high recall rate. In early 
search engine that offered disambiguation to search terms. 
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User intention identification plays an important role in the 
intelligent semantic search engine. 

B. Proposed system 

We propose the semantic web based search engine which is 
also called as Intelligent Semantic Web Search Engines. 

We use the power of xml meta-tags deployed on the web page 
to search the queried information. The xml page will be 
consisted of built-in and user defined tags. Here propose the 
intelligent semantic web based search engine. We use the 
power of xml meta-tags deployed on the web page to search 
the queried information. The xml page will be consisted of 
built-in and user defined tags. The metadata information of 
the pages is extracted from this xml into rdf. our practical 
results showing that proposed approach taking very less time 
to answer the queries while providing more accurate 
information. 


C. Re -ranking framework 
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Figure 1 . The conventional image re-ranking framework 


Major internet image search engines have since adopted the 
re-ranking strategy. Its diagram is shown in Figure 1 . Given a 
query keyword input by a user, according to a stored 
word-image index file, a pool of images relevant to the query 
keyword are retrieved by the search engine. By asking a user 
to select a query image, which reflects the user’s search 
intention, from the pool, the remaining images in the pool are 
re-ranked based on their visual similarities with the query 
image. The visual features of images are pre-computed offline 
and stored by the search engine. The main online 
computational cost of image re-ranking is on comparing 
visual features. In order to achieve high efficiency, the visual 
feature vectors need to be short and their matching needs to be 
fast. Another major challenge is that the similarities of 
lowlevel visual features may not well correlate with images 
high-level semantic meanings which interpret users search 
intention. To narrow down this semantic gap, for offline 
image recognition and retrieval, there have been a number of 
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studies to map visual features to a set of predefined concepts 
or attributes as semantic signature. However, these approaches 
are only applicable to closed image sets of relatively small 
sizes. 

D. New image re -ranking framwork 


considerations, keyword expansions are found in a 
search-and-rank way as follows. 

For each image I belongs to S(q), all the images in S(q) are 
reranked according to their visual similarities 
to I. 



Figure 2: Diagram of our new image re-ranking framework 

The diagram of our approach is shown in Figure 2. At the 
offline stage, the reference classes (which represent different 
semantic concepts) of query keywords are automatically 
discovered. For a query keyword (e.g. “apple”), a set of most 
relevant keyword expansions (such as “red apple”, “apple 
macbook”, and “apple iphone”) are automatically selected 
considering both textual and visual information. This set of 
keyword expansions defines the reference classes for the 
query keyword. In order to automatically obtain the training 
examples of a reference class, the keyword expansion (e.g. 
“red apple”) is used to retrieve images by the search engine. 
Images retrieved by the keyword expansion (“red apple”) are 
much less diverse than those retrieved by the original 
keyword (“apple”). After automatically removing outliers, the 
retrieved top images are used as the training examples of the 
reference class. Some reference classes (such as “apple 
laptop” and “apple macbook”) have similar semantic 
meanings and their training sets are visually similar. In order 
to improve the efficiency of online image re-ranking, 
redundant reference classes are removed. At the online stage, 
a pool of images are retrieved by the search engine according 
to the query keyword input by a user. Since all the images in 
the pool are relevant to the query keyword, they all have 
pre-computed semantic signatures in the semantic space of the 
query keyword. Once theuser chooses a query image, all the 
images are re-ranked by comparing similarities of the 
semantic signatures. 

E. Keyword expansion 

For a keyword q, we automatically define its reference 
classes through finding a set of keyword expansions E(q) 
most relevant to q. To achieve this, a set of images S(q) 
are retrieved by the search engine using q as query based on 
textual information. Keyword expansions are found from the 
words extracted from the images in S(q)3. A keyword 
expansion e 2 Eq is expected to frequently appear in S(q). In 
order for reference classes to well capture the visual content 
of images, we require that there is a subset of images which all 
contain e and have similar visual content. Based on these 


III. SUMMERY OF TECHNOLOGY 
A. EXPERIMENTAL RESULTS 

The images for testing the performance of re -ranking and the 
images of reference classes can be collected at different time 
and from different search engines. Given a query keyword, 
1000 images are retrieved from the whole web using certain 
search engine. As summarized in Table 1, we create three data 
sets to evaluate the performance of our approach in different 
scenarios. In data set I, 120; 000 testing images for re-ranking 
were collected from the Bing Image Search using 120 query 
keywords in July 2010. These query keywords cover diverse 
topics including animal, plant, food, place, people, event, 
object, scene, etc. The images of reference classes were also 
collected from the Bing Image Search around the same time. 
Data set II use the same testing images for re-ranking as in 
data set I. However, its images of reference classes were 
collected from the Google Image Search also in July 2010. In 
data set III, both testing images and images of reference 
classes were collected from the Bing Image Search but at 
different time (eleven months apart)5. All testing images for 
re-ranking are manually labeled, while images of reference 
classes, whose number is much larger, are not labeled. 
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Figure 4: Comparison 
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B. ONLINE EFFICIENCY 


The online computational cost of image re-ranking depends 
on the length of visual feature (if directly comparing visual 
features) or semantic signatures (if using our approach). In 
our experiments, the visual features have around 1; 700 
dimensions, and the averaged number of reference classes per 
query is 25. Therefore the length of the single semantic 
signature (QSVSS Single) is 25 on average. Since six types of 
visual features are used, the length of the multiple semantic 
signatures (QSVSS Multiple) is 150. It takes 12ms to re-rank 
1000 images matching the visual features, while QSVSS 
Multiple and QSVSS Single only need 1:14ms and 0:2ms 
respectively. Given the large improvement of precisions our 
approach has achieved, it also improves the efficiency by 
around 10 to 60 times compared with matching visual features 
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Figure 4: Comparisons of averaged top m precisions of re-ranking 
images outside reference classes and using universal semantic 
space on data set. 


C. RE-RANKING IMAGES O UTSIDE THE REFERENCE 
CLASS 

It is interesting to know whether the learned queryspecific 
semantic spaces are effective for query imageswhich are 
outside the reference classes. To answer this question, if the 
category of an query image corresponds to a reference class, 
we deliberately delete this reference class and use the 
remaining reference classes to train SVM classifiers and to 
compute semantic signatures when comparing this query 
image with other images. We repeat this for every image and 
calculate the average top m precision. Multiple semantic 
signatures (QSVSS Multiple) are used. The results are shown 
in It still greatly outperforms the approaches of directly 
comparing visual features. This result can be explained from 
two aspects Many negative examples (images belonging to 
different categories than the query image) are well modeled 
by the reference classes and are therefore pushed backward on 
the ranking list. 

IV. CONCLUSION 

We propose a novel image re -ranking framework, which 
learns query-specific semantic spaces to significantly improve 
the effectiveness and efficiency of online image reranking. 
The visual features of images are projected into their related 
visual semantic spaces automatically learned through 
keyword expansions at the offline stage. The extracted 
semantic signatures can be 70 times shorter than the original 
visual feature on average, while achieve 20%35% relative 
improvement on re-ranking precisions over state-ofthe-art 
methods. 

V. FUTURE WORK 

In future, we can work on improving the quality of 
re-ranked images, we intent to combine this work with photo 
quality assessment work in to re-rank images not only by 
content similarity but also by the visual quality of the images. 
Also, this framework can be further improved by making use 
of the query log data, which provides valuable co-occurrence 
information of keywords, for keyword expansion. 
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